Monitoring system for a photography unit, monitoring method, computer program, and storage medium

ABSTRACT

A coverage area picture imaging a maximum area is displayed on a coverage area picture display. A camera is moved within a predetermined range and a plurality of frames obtained as a result form a picture. The pixels of the picture is decimated in vertical and horizontal directions and form a thumbnail as the coverage area picture. The coverage area picture display presents a display indicating a direction in which the camera is currently directed for picture photographing. In accordance with the display, a photographing direction is controlled. A plurality of frames are photographed with respect to a designated position, then stored, and displayed. A whole picture display presents a whole panorama picture. A selective picture display presents a frame at the position designated within the whole panorama picture as a selected picture.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a monitoring system, monitoring method,computer program and storage medium for use with a surveillance camera.

2. Description of the Related Art

Monitoring systems for monitoring a wide area are conventionally used.For example, a monitoring system may be used for surveillance of sea andriver regions, monitoring of trespassers, monitoring of the behavior ofwild animals, and for other purposes. A video camera having a largenumber of pixels is used to capture the image of a wide area. For thisreason, the cost of the system typically becomes higher. A technique hasbeen proposed which captures a still picture by shifting successivelycapture areas from one to another and then linking the still pictures togenerate a picture of the area to be monitored. The whole picture has anextremely high resolution. When an expanded picture of one portion ofthe whole picture is obtained, the resolution of the expanded picture isstill high and a clear image thus results.

To capture the picture of a wide area, the number of still picturesforming the whole picture of the wide area increases. Time required tocapture still frame pictures forming the whole picture is prolonged. Inpractice, an area to be monitored is typically limited. The monitoringsystem is preferably usable in a dark environment under which the nakedeye of the human is unable to see objects. Using an infrared camera, themonitoring system may have a dark vision feature. However, capturedimage is typically dark and unclear to identify. The operability of themonitoring system is not satisfactory because of image darknessparticularly when the user attempts to direct the camera to a desireddirection while viewing the captured picture, or when the user attemptsto expand an arbitrary point or area in the captured picture.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to provide amonitoring system, monitoring method, computer program and storagemedium for displaying a whole picture to be monitored, andretrospectively reproducing and displaying past frames with respect toan arbitrary position.

It is another object of the present invention to provide a monitoringsystem, monitoring method, computer program and storage mediumappropriate for monitoring an actually desired area, and easily operatedto control the direction of a camera under a dark environment thatresults in a dark picture.

In a first aspect of the present invention, a monitoring system includesa picture photographing unit for photographing a picture, aphotographing direction varying unit for varying a photographingdirection of the picture photographing unit, a storage unit for storingpicture data, a picture display unit, and a controller which stores, inthe storage unit, one of a source picture including a. plurality ofstill frame pictures photographed in the photographing directions withina predetermined coverage area within a predetermined range of thephotographing direction varying unit and a picture which is obtained bycompressing the source picture, and displays, on the picture displayunit, a whole panorama picture generated from the one of the sourcepicture and the compressed picture, wherein a picture within thepredetermined coverage area is photographed with the picturephotographing direction varied, the coverage area picture is displayedon the picture display unit, the photographing direction is controlledto a desired position by designating the desired position within thecoverage area picture, and the whole panorama picture captured withrespect to the designated position is displayed on the picture displayunit.

In a second aspect of the present invention, a monitoring method forstoring one of a source picture including a plurality of still framepictures photographed in photographing directions within a predeterminedcoverage area within a predetermined range of a photographing directionvarying unit varying a photographing direction of a picturephotographing unit, and a picture which is obtained by compressing thesource picture, and for displaying a whole panorama picture generatedfrom the one of the source picture and the compressed picture, includesthe steps of photographing a coverage area picture with thephotographing direction varied to display the coverage area picture, andcontrolling the photographing direction to a desired position bydesignating the desired position within the coverage area picture todisplay the whole panorama picture photographed with respect to thedesignated position.

In a third aspect of the present invention, a computer executableprogram for storing one of a source picture including a plurality ofstill frame pictures photographed in photographing directions within apredetermined coverage area within a predetermined range of aphotographing direction varying unit varying a photographing directionof a picture photographing unit, and a picture which is obtained bycompressing the source picture, and for displaying a whole panoramapicture generated from the one of the source picture and the compressedpicture, includes program codes for performing the steps ofphotographing a coverage area picture with the photographing directionvaried to display the coverage area picture, and controlling thephotographing direction to a desired position by designating the desiredposition within the coverage area picture to display the whole panoramapicture captured with respect to the designated position.

In a fourth aspect of the present invention, a computer readable storagemedium stores a computer executable program for storing one of a sourcepicture including a plurality of still frame pictures photographed inphotographing directions within a predetermined coverage area within apredetermined range of a photographing direction varying unit varying aphotographing direction of a picture photographing unit, and a picturewhich is obtained by compressing the source picture, and for displayinga whole panorama picture generated from the one of the source pictureand the compressed picture. The computer executable program includesprogram codes for performing the steps of photographing a coverage areapicture with the photographing direction varied to display the coveragearea picture, and controlling the photographing direction to a desiredposition by designating the desired position within the coverage areapicture to display the whole panorama picture captured with respect tothe designated position.

The period of time required to capture the whole panorama picture isprevented from being prolonged because the picture photographing unit isnot fully moved within the predetermined range. Since the coverage areapicture with the picture photographing unit fully moved within thepredetermined range is displayed, the photographing direction to obtainthe picture of a desired area is easily set. Even if the picture beingphotographed is dark, the photographing direction is easily set. Theoperability of the system is improved.

In a fifth aspect of the present invention, a monitoring system includesa picture photographing unit for photographing a picture, aphotographing direction varying unit for varying a photographingdirection of the picture photographing unit, a storage unit for storingpicture data, a picture display unit, and a controller which stores inthe storage unit, one of a source picture including a plurality of stillframe pictures photographed in the photographing directions within apredetermined coverage area within a predetermined range of thephotographing direction varying unit and a picture which is obtained bycompressing the source picture, and displays, on the picture displayunit, a whole panorama picture generated from the one of the sourcepicture and the compressed picture, wherein an arbitrary point of thepicture display unit is indicated, only a still frame picture at theindicated arbitrary point is photographed by the picture photographingunit, and the photographed still frame picture is displayed on thepicture display unit at a predetermined position thereof.

In a sixth aspect of the present invention, a monitoring system includesa picture photographing unit for photographing a picture, aphotographing direction varying unit for varying a photographingdirection of the picture photographing unit, a storage unit for storingpicture data, a picture display unit, and a controller which stores, inthe storage unit, one of a source picture including a plurality of stillframe pictures photographed in the photographing directions within apredetermined coverage area within a predetermined range of thephotographing direction varying unit and a picture which is obtained bycompressing the source picture, and displays, on the picture displayunit, a whole panorama picture generated from the one of the sourcepicture and the compressed picture, wherein an arbitrary point of thepicture display unit is indicated, only a still frame picture at theindicated arbitrary point is read from one of the source picture and thecompressed picture stored in the storage unit, and the read still framepicture is displayed on the picture display unit at a predeterminedposition thereof.

In a seventh aspect of the present invention, a monitoring method forstoring one of a source picture including a plurality of still framepictures photographed in photographing directions within a predeterminedcoverage area within a predetermined range of a photographing directionvarying unit varying a photographing direction of a picturephotographing unit, and a picture which is obtained by compressing thesource picture, and for displaying a whole panorama picture generatedfrom the one of the source picture and the compressed picture, includesthe steps of indicating an arbitrary point within the whole panoramapicture, photographing only a still frame picture at the indicatedarbitrary point, and displaying the photographed still frame picture inthe whole panorama picture at a predetermined position therewithin.

In an eighth aspect of the present invention, a computer executableprogram for storing one of a source picture including a plurality ofstill frame pictures photographed in photographing directions within apredetermined coverage area within a predetermined range of aphotographing direction varying unit varying a photographing directionof a picture photographing unit, and a picture which is obtained bycompressing the source picture, and for displaying a whole panoramapicture generated from the one of the source picture and the compressedpicture, includes program codes for performing the steps of indicatingan arbitrary point within the whole panorama picture, photographing onlya still frame picture at the indicated arbitrary point, and displayingthe photographed still frame picture in the whole panorama picture at apredetermined position therewithin.

In a ninth aspect of the present invention, a computer executableprogram for storing one of a source picture including a plurality ofstill frame pictures photographed in photographing directions within apredetermined coverage area within a predetermined range of aphotographing direction varying unit varying a photographing directionof a picture photographing unit, and a picture which is obtained bycompressing the source picture, and for displaying a whole panoramapicture generated from the one of the source picture and the compressedpicture, includes program codes for performing the steps of indicatingan arbitrary point within the whole panorama picture, reading only astill frame picture at the indicated arbitrary point from the storedsource pictures and the stored compressed pictures, and displaying theread still frame picture in the whole panorama picture at apredetermined position therewithin.

In a tenth aspect of the present invention, a storage medium stores acomputer executable program for storing one of a source pictureincluding a plurality of still frame pictures photographed inphotographing directions within a predetermined coverage area within apredetermined range of a photographing direction varying unit varying aphotographing direction of a picture photographing unit, and a picturewhich is obtained by compressing the source picture, and for displayinga whole panorama picture generated from the one of the source pictureand the compressed picture. The computer executable program includesprogram codes for performing the steps of indicating an arbitrary pointwithin the whole panorama picture, photographing only a still framepicture at the indicated arbitrary point, and displaying thephotographed still frame picture in the whole panorama picture at apredetermined position therewithin.

In an eleventh aspect of the present invention, a storage medium storesa computer executable program for storing one of a source pictureincluding a plurality of still frame pictures photographed inphotographing directions within a predetermined coverage area within apredetermined range of a photographing direction varying unit varying aphotographing direction of a picture photographing unit, and a picturewhich is obtained by compressing the source picture, and for displayinga whole panorama picture generated from the one of the source pictureand the compressed picture. The computer executable program includesprogram codes for performing the steps of indicating an arbitrary pointwithin the whole panorama picture, reading only a still frame picture atthe indicated arbitrary point from the stored source pictures and thestored compressed pictures, and displaying the read still frame picturein the whole panorama picture at a predetermined position therewithin.

Since an optical axis of the picture photographing unit is directed tothe center of a still frame picture at an arbitrary point, the stillframe picture at the arbitrary point is photographed and displayed whilethe whole panorama picture is displayed at the same time. Since a stillframe picture at an arbitrary point is reproduced from already storeddata, a still frame picture at an arbitrary point is reproduced anddisplayed in retrospect while the whole panorama picture is displayed atthe same time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram diagrammatically illustrating a monitoringsystem in accordance one embodiment of the present invention;

FIG. 2 is a block diagram of the embodiment of the present invention;

FIG. 3 diagrammatically illustrates a display screen in accordance withthe embodiment of the present invention;

FIG. 4 diagrammatically illustrates a select display screen inaccordance with the embodiment of the present invention;

FIG. 5 diagrammatically illustrates a recorded data display screen whichis reproduced in accordance with the embodiment of the presentinvention;

FIG. 6 diagrammatically illustrates photographing and picture capturingoperations in accordance with the embodiment of the present invention;

FIG. 7 is a diagram illustrating a range to an object, photographingarea, and resolution in accordance with the embodiment of the presentinvention;

FIGS. 8A and 8B illustrate a management method of photographed pictures;

FIG. 9 is a flow diagram illustrating a capturing operation of acoverage area picture in accordance with the embodiment of the presentinvention;

FIG. 10 is a flow diagram illustrating a displaying operation of thecoverage area picture in accordance with the embodiment of the presentinvention;

FIG. 11 is a flow diagram illustrating a capturing operation of thecoverage area picture in accordance with the embodiment of the presentinvention;

FIG. 12 is a flow diagram illustrating a capturing operation anddisplaying operation of a selected picture in accordance with theembodiment of the present invention;

FIG. 13 is a flow diagram illustrating a capturing operation of a frameof a whole picture in accordance with the embodiment of the presentinvention;

FIG. 14 is a flow diagram illustrating a reproduction operation ofstored picture data in accordance with the embodiment of the presentinvention;

FIG. 15 is a flow diagram illustrating a capturing operation of oneframe only from a photographing unit in accordance with the embodimentof the present invention; and

FIG. 16 is a flow diagram illustrating an operation in which one frameonly is reproduced from stored picture data in accordance with theembodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

One embodiment of the present invention will now be discussed withreference to the drawings. FIG. 1 is a block diagram diagrammaticallyillustrating a monitoring system in accordance one embodiment of thepresent invention. A computer 1, connected to a display 2, controls acamera unit 3. In the system shown in FIG. 1, the single computer 1controls two camera units 3, and another computer 1′, connected toanother display 2′, controls another camera unit 3′. In this way, asingle computer controls a plurality of camera units 3.

The camera unit 3 is integrally formed of a pan and tilt section 4 andcamera section 5. The camera unit 3 is mounted so that a remote targetarea is photographed. As an example, the camera section 5 has atelephoto lens with a magnification of 10 or 70, and takes a picture ofan area several tens of meters to several kilometers away.

The camera section 5 is a digital still camera, which is turned on insynchronization with an external trigger. The image pickup device of thecamera section 5, for example, a CCD (Charge-Coupled Device), has aresolution of 640×480 pixels (Video Graphics Array, VGA), resolution of1024×768 pixels (eXtended Graphics Array, XGA), resolution of 1208×1024pixels (Super eXtended Graphics Array, SXGA) or the like. If a VGA imagepickup device is used, picture data is output at a rate of 30 fps(frames/second). If an XGA image pickup device is used, picture data isoutput at a rate of 15 fps. If an SXGA image pickup device is used,picture data is output at a rate of 7.5 fps.

Video data is transferred from the camera unit 3 to the computer 1through a bus 6. The bus 6 allows the video data and a control signal ofthe camera unit 3 to be transferred therethrough. The above-discussedconstruction is also applied to the computer 1′ and camera unit 3′.

The computers 1 and 1′ store video data from the camera units 3 and 3′,respectively. As will be discussed later, the computers 1 and 1′ includeGUI (Graphical User Interface) to control the camera units 3 and 3′respectively to photograph a target area desired by the user. The videodata is compressed in accordance with JPEG (Joint Photographic ExpertsGroup).

The computers 1 and 1′ are mutually interconnected to each other througha LAN (Local Area Network). Another computer 8 is connected to the LAN7. A display 9 is connected to the computer 8. The computer 8 receivespicture data from the computers 1 and 1′ through the LAN 7, stores thepicture data in an archive 10, and processes the picture data. Forexample, the computer 8 performs face recognition, baggage recognition,environment recognition, vehicle recognition, etc. on the picture data.Like a tape streamer, the archive 10 stores a vast amount of data.

FIG. 2 illustrates the computer 1 and camera unit 3 in the monitoringsystem more in detail. As shown, components of the camera unit 3 and thecomputer 1 are connected to a controller bus 21.

The pan and tilt section 4 includes a pan part 4 a and a tilt part 4 b.The pan part 4 a and tilt part 4 b have respective sources of power suchas stepping motors, and respectively pans and tilts the camera section 5in response to a control signal which is supplied from a CPU (CentralProcessing Unit) 33 through the controller bus 21. The camera section 5is mounted on the pan and tilt section 4. A panning operation refers toa movement in which a camera pans in a horizontal direction and atilting operation refers to a movement in which the camera is verticallytilted. For example, a maximum pan angle is 180° and a maximum tiltangle is 50°.

As will be discussed later, the camera section 5 is movable within themaximum tilt angle range of ±15° and the maximum pan angle range of±50°. Each time the center of a picture is shifted by an angle of view,the shutter of the camera section 5 is turned on to photograph stillpictures (also simply referred to as frames). For example, a total ofM×N frames (=8×16=128 frames, for example), namely, M frames (8 frames,for example) in a vertical direction and N frames (16 frames, forexample) in a horizontal direction, are successively photographed,compressed, and then linked together to form a single whole picture. Forexample, each frame is an XGA (1024×768 pixels) picture. The total of128 frames forms a picture of about 100 million pixels (16,384 (1024×16)pixels in a horizontal direction and 6,144 (768×8) pixels in a verticaldirection), if an overlapping coverage is disregarded. It takes aboutfive seconds for the system to take 128 frames. The overlapping coverageis typically 16 pixels in a vertical direction and 16 pixels in ahorizontal direction.

The camera section 5 is a digital still camera, and includes a lens unit22, focus-zoom-iris controller 23, and photographing unit 24. Thefocus-zoom-iris controller 23 is controlled by a control signal which issupplied by the controller CPU 33 through the controller bus 21. Thephotographing unit 24 includes a solid-state image pickup device such asa CCD and a camera signal processing circuit. A digital video signalfrom the photographing unit 24 is written onto a buffer memory 26through an interface 25 complying with the IEEE (Institute of Electricaland Electronics Engineers) 1394 Standard.

The output data of the buffer memory 26 is fed to a JPEG encoder andmetadata attacher 27. The JPEG encoder and metadata attacher 27 convertspicture data into JPEG data. The JPEG defines one method of datacompression. The picture data may be compressed using another method ormay not be compressed.

The camera unit 3 includes a GPS (Global Positioning System) receiver 28to acquire a position fix. With the GPS receiver 28, the position dataof the camera unit 3 is stored, and the direction of the camera isdetected. The directions of a plurality of cameras are thus controlledin an interlocking motion. The GPS receiver 28 is controlled by acontrol signal which is supplied by the controller CPU 33 through thecontroller bus 21.

The output signal of the GPS receiver 28 is fed to a metadata generator29. The metadata generator 29 generates position information(information such as latitude and longitude, bearing, and altitude)based on the position fix provided by the GPS receiver 28, and metadata(time and parameters of the camera section 5 such as magnification,focus value, and iris value). The position information and metadata arefed to the JPEG encoder and metadata attacher 27. The JPEG encoder andmetadata attacher 27 attaches the position information and metadata tothe JPEG data.

The JPEG data, and the position information and metadata attachedthereto are stored in a main memory 30 such as a hard disk, while beingsupplied to a graphic controller 31 and image compressor 32 at the sametime. In this specification, the accumulation of data in the main memory30 is referred to as “recording”, and the reading of data from the mainmemory 30 is referred to as “reproduction”. Also in this specification,an operation in which a picture currently being photographed isdisplayed without being stored in the main memory 30 is referred to as alive mode, and an operation in which data stored in the main memory 30is reproduced and displayed is referred to as a view mode.

The main memory 30 has a function as a server. For example, the amountof data of a single frame as a result of compressing an XGA picturebecomes about 100 Kbytes, and a picture of 128 frames becomes 12.5Mbytes. If the main memory 30 has a capacity of 80 Gbytes, it can holdJPEG data for full one day long recording. The view mode enables thereproduction of not only data stored in the main memory 30 but alsoolder data stored in a storage device such as an archive.

The JPEG data read from the main memory 30 is then supplied to thegraphic controller 31. The image compressor 32 generates a compressedpicture or a thumbnail from one of the JPEG data from the JPEG encoderand metadata attacher 27 and the JPEG data read from the main memory 30.For example, by decimating the pixels in a vertical direction and in ahorizontal direction, a whole panorama picture is generated. The imagecompressor 32 also performs a compression process to form a coveragearea picture to be discussed later. In the case of the XGA picture, awhole panorama picture of 400×1000 pixels is produced when the data ofabout 100 million pixels is JPEG compressed and then processed by theimage compressor 32 JPEG. The coverage area picture is a thumbnail, andis an image even coarser than the whole panorama picture.

The graphic controller 31 performs a graphic process to convert the JPEGdata into bitmap data and to present a desired display on the screen ofthe display 2. Specifically, GUI displays such as a coverage areapicture display, whole picture display, selected picture display, andbuttons are presented on the screen of the display 2. The detail of thedisplay will be discussed later.

The graphic controller 31 performs image processing, thereby detecting achange in the picture. A change in the picture is the one that occurswith respect to a reference picture. For example, in the view mode, acurrent picture is compared with the reference picture stored before,and a change in the picture is detected. A picture at a predeterminedtime on the preceding day is set as a reference picture, and a picturedifference between the reference picture and the picture storedsubsequent to that point of time is detected. If the absolute value ofthe picture difference becomes equal to or rises above a predeterminedvalue, the change is accepted as a picture change. In one method ofdetecting a difference, a difference between the pixels at the samespatial position from the reference picture to the picture to becompared is detected. Instead of detecting difference for all pixels,representative pixels or remaining pixels subsequent to decimation maybe subjected to difference calculation. The difference calculation maybe performed for a particular color to detect a change in an objecthaving the predetermined color.

If a change is detected, a display alarm is provided on the screen ofthe display 2, thereby distinguishing from the remaining frames the onewhich has the change. Specifically, the display alarm is provided usinga luminance change, color change, or display blinking. Any predeterminedpicture may be selected from among stored pictures as the referencepicture.

As discussed above, the controller CPU 33 connected to the controllerbus 21 performs lens control of the camera section 5 (for focusing, forexample), exposure control (for stop, gain, and electronic shutterspeed, for example), white balance control, and image quality control,while also controlling the pan part 4 a and tilt part 4 b.

An I/O (input/output) port 34 connects to a keyboard 35 and mouse 36. Amemory card 37 and clock 38 are respectively connected to the I/O port34. The JPEG data, and the position information and metadata attachedthereto, stored in the main memory 30, are written onto the memory card37. Time data is acquired from the clock 38.

FIG. 2 shows units connected to the controller bus 21. The camera unit 3may be installed at a location remote from the computer 1, and bothunits may be connected through an IEEE1394 or USB interface. In thiscase, an optical fiber may serve as a physical transmission line. Theuse of the optical fiber allows the camera unit 3 to be installedseveral hundred meters to several kilometers away from the computer 1.Furthermore, the two units may be interconnected using a radio LAN(Local Area Network).

FIG. 3 diagrammatically illustrates a GUI display screen in accordancewith the embodiment of the present invention. Operation buttons anddisplay regions provided on the GUI screen are discussed. One singlescreen includes a coverage area picture display 101, whole picturedisplay 102, and selected picture display 103.

The coverage area picture display 101 presents a coverage area picture.The coverage area picture is a picture which is photographed by thecamera unit 3 in the maximum photographing area thereof, and is composedof a plurality of frames. As already discussed, the maximum pan angle is180°, and the maximum tilt angle is 50°. The coverage area picture isformed of a plurality of frames photographed in these maximum ranges.For example, the camera unit 3 is mounted, and the camera section 5 ismoved with the optical axis thereof shifted within the maximum ranges. Apicture is formed of a plurality of frames obtained as a result. Thepixels forming the picture are then decimated in vertical and horizontaldirections to be thumbnail. The resulting thumbnail is the coverage areapicture.

The coverage area picture display 101 indicates a current position ofthe lens optical axis of the camera unit 3 (camera live position) at anintersection of a line segment 101 a and line segment 101 b. By movingthe line segments 101 a and 101 b, a desired position is designatedwithin the coverage area picture, and a picture photographing directionis thus controlled to direct to the designated position. With thedesignated position being set as a center or home position, M×N stillframe pictures are photographed within the predetermined ranges, andstored, or displayed. The present invention is not limited to the linesegments,101 a and 101 b. Alternatively, a pointer or mouse 36 may pointto any position on the screen presented on the coverage area picturedisplay 101, and the camera unit 3 may be controlled so that the lensoptical axis of the camera unit 3 is directed to the designatedposition.

The whole picture display 102 presents a whole panorama picture. Thewhole panorama picture is the one into which the image compressor 32compresses the JPEG data corresponding to a source picture photographed.A monitoring operation is performed watching the displayed wholepanorama picture. As already discussed, if a picture change is detected,the system provides an alarm display in which a frame within which thechange is detected is displayed in a manner different from the remainingframes in the whole picture presented on the whole picture display 102.

The selected picture display 103 presents a selected picture. Theselected picture is an expanded image of a portion of the whole panoramapicture. An expanded image is presented by displaying an uncompressedsource frame image. The image is further expanded using digital signalprocessing.

An EXIT button 104 is used to cut off power to the monitoring system. Acamera system OFF button 105 is used to cut off power to the camera unit3.

A VIEW MODE button 106 is used to switch the mode of the monitoringsystem to a view mode. During the vide mode, the whole picture andpartial picture are displayed based on the picture data stored in themain memory 30 or in another server.

A LIVE MODE button 107 is used to switch the mode of the monitoringsystem to the live mode. During the live mode, the whole picture andpartial picture are displayed based on the frames currently beingphotographed by the camera unit 3.

A compass display region 108 is used to display a bearing to which theoptical axis of the lens of the camera is directed. A GPS data displayregion 109 displays the latitude, longitude, and altitude where thecamera unit 3 is installed, and date and time at which the photographingoperation is performed. Data shown on the regions 108 and 109 is the onethat is acquired by the GPS receiver 28 in the camera unit 3 in theposition fixing operation thereof. A view offset button 110 is used toadjust the position of a selected frame. The view offset button 110moves the single frame, selected by a pointer in the whole picturepresented by the whole picture display 102, upward, downward, to theleft or to the right. A plurality of frames forming the whole pictureare linked together with one frame overlapping the next by apredetermined number of pixels, 16 pixels, for example. By moving eachframe within the overlap coverage, adjacent frame alignment is assured.The linking condition between the adjacent frames is thus smoothed.

A mode display region 129 is used to display mode information, alarminformation, error information, etc. The mode information informs theuser of the mode of the monitoring system, and specifically, the modeinformation indicates the live mode or the view mode. The alarminformation alerts the user and, for example, the alarm information isprovided when the frame reaches a limit with the view offset button 110being pressed. The error information informs the user of an erroroccurring in the monitoring system.

A camera control region 111 includes a ZOOM button 112, FOCUS button113, IRIS button 114, camera configuration button 115, and white balancebutton 116. The ZOOM button 112 adjusts the zoom of the camera unit 3.The FOCUS button 113 adjusts the focus of the camera unit 3. The IRISbutton 114 adjusts the iris of the camera unit 3. The cameraconfiguration button 115 adjusts γ characteristics, shutter speed, andgain of the camera unit 3. The white balance button 116 adjusts thewhite balance of the camera unit 3. While the monitoring system is inthe view mode, the display of the camera control region 111 may beomitted.

A SELECT button 117 is used to display a select display in the viewmode. The select display is used to identify an area, desired to bereproduced or stored, by a frame constituting the whole picture.

FIG. 4 diagrammatically illustrates a select display screen inaccordance with the embodiment of the present invention. As shown, theselect display includes a closing button 151, display screen 152, andclosing button 153. The close buttons 151 and 153 are clicked to closethe select display. The display screen 152 presents a whole picturepresented on the whole picture display 102, and indicates an outline ofa frame to be captured. The whole picture displayed on the whole picturedisplay 102 may be partitioned according to unit of frames to becaptured, and may then be displayed on the display screen 152. A grid oflines may be superimposed on the whole picture. If the pointer ispointed to any position on a desired picture, the frame indicated bythat point is selected, and one of the brightness, resolution, andcontrast of the indicated frame varies to show that the frame isselected.

A REC MODE selection menu 118 is a pull-down menu to select a recordingmode. The pull-down menu displays a recording mode which represents acombination of a picture size to be recorded and recording method (RUNor SINGLE). The picture size can be any of a whole picture formed of8×16 frames, a partial picture formed of selected 4×8 frames of thewhole picture, and a partial pictured formed of selected 2×4 frames ofthe whole picture. The partial picture is the one at a position selectedon the select display. The RUN recording method is used to record thephotographed picture generated every predetermined period of time (everyfive seconds, for example), and the SINGLE recording method is used torecord the photographed picture once. The recording mode is used toselect a combination of the RUN recording method and SINGLE recordingmethod.

A stage configuration button 119 is a fine adjustment button to adjustthe accuracy with which a stage of the camera unit 3 is moved. A messageregion 120 is used to display a connection status between the controlcomputer 1 and camera unit 3, and a control status of the stage of thecamera unit 3. If the control computer 1 is connected to the camera unit3, a message reading “IMAGE SERVER CONNECT” is posted on the messageregion 120 as shown in FIG. 3. When the stage of the camera unit 3 is ina controllable state, a message reading “STAGE CONTROL ACTIVE” is postedon the message region 120.

A REC button 121 starts the recording of the picture. If the REC button121 is designated by the pointer, the recording corresponding to therecording mode selected in the REC MODE selection menu 118 starts.Specifically, the recording corresponding to a mode selected from amongthe modes RUN (8×16), RUN (4×8), RUN (2×4), SELECT SINGLE RUN (8×16),SELECT SINGLE RUN (4×8), SELECT SINGLE RUN (2×4), etc. starts.

A PLAY button 122 is used to reproduce the picture data stored in theserver (main memory 30). Specifically, if the PLAY button 122 isdesignated, a recorded data display screen is presented. Information toidentify stored picture data appears on the recorded data displayscreen. The information is based on information described in a directionfile to be discussed later.

FIG. 5 illustrates one example of the recorded data display screen.Shown on the recorded data display screen are a minimizing button 161,maximizing button 162, closing button 163, date box 164, time box 165,recorded data display area 166, updated data display area 167, OK button168, cancel button 169, and storage device switching button 170.

The minimizing button 161 is clicked to minimize the size of therecorded data display screen to icons. The maximizing button 162 isclicked to maximize the size of the recorded data display screen overthe full screen of the monitor. The closing button 163 is clicked toclose the recorded data display screen.

The date box 164 is used to designate the date of the recorded data tobe displayed on the whole picture display 102. For example, click abutton 164 a arranged on the right hand end of the date box 164, and alist of the dates of displayable recorded data appears in a pull-downmenu form. Date is selected from among the listed dates.

The time box 165 is used to designate the time of the recorded data tobe displayed on the whole picture display 102. For example, click abutton 165 a arranged on the right hand end of the time box 165, and alist of the times of displayable recorded data appears in a pull-downmenu form. Time is selected from among the listed times.

The recorded data display area 166 shows, from the storage device,recorded data matching the date and time designated by the date box 164and time box 165. The updated data display area 167 shows latestrecorded data from the recorded data stored in the storage device.Alternatively, the latest recorded data from among the recorded datadesignated by the date box 164 and time box 165 may be displayed.

The OK button 168 is clicked subsequent to the designation of thedesired recorded data. The cancel button 169 is clicked to close therecorded data display screen. The storage device switching button 170 isused to enter a check mark to switch the destination of data storagefrom the storage device to a detachable semiconductor memory card, forexample.

Returning to FIG. 3, a STOP button 123 is used to stop the recording orreproduction of the data. The STOP button 123 may be presentedsubsequent to the designation of the REC button 121 or the PLAY button122 by the pointer.

A set camera center POSITION button 124 is used to designate thedirection of the camera as the center of the picture (8×16 frames).

A HOME button 125 is used to control the camera unit 3 to direct theoptical axis of the lens of the camera unit 3 to a home position. Thehome position refers to a position where the camera is directed to theleftmost position. A LIVE/VIEW POSITION button 126 is used to pan ortilt the camera.

ZOOM buttons 127A and 127B are used to zoom out and in the selectedpicture displayed on the selected picture display 103.

A MAX VIEW button 128 is used to expand and display the selected pictureon a different display such as the whole picture display 102.

A production method of the whole picture in accordance with theembodiment of the present invention will be discussed with reference toFIG. 6. As shown, the camera section 5 is mounted on the panhead of thepan and tilt section 4 in the camera unit 3, and the photographingdirection is varied from the home position of the camera. When viewedfrom the camera side, photographed frames of M rows and N columns aresuccessively numbered. Specifically, the rows from top to bottom arerespectively numbered with 1, 2, . . . , M, and the columns from rightto left are respectively numbered with 1, 2, . . . , N. The homeposition is a position where the frame at coordinates (1,1) isphotographed.

If the frame at coordinates (1,1) is photographed, the camera unit 3 istilted downward to photograph the frame at coordinates (2,1). Insuccession, the frame (3,1), . . . , (M,1) are successivelyphotographed. Next, the frame at the top row and second column atcoordinates (1,2) is photographed. The photographing operation continuesuntil the frame at coordinates (M, N) is photographed. As alreadydescribed, there is an overlap coverage of 16 pixels between one frameand a next frame adjacent thereto. The photographed frame is JPEGcompressed, and stored in the main memory 30.

In the case of the XGA picture (having 1024×768 pixels), the total of128 frames form a picture of about 100 million pixels (1024×16 (=16,384)pixels in a horizontal direction and 768×8 (=6,144) pixels in a verticaldirection), if an overlapping coverage is disregarded. The whole picturedisplay 102 shows a compressed picture or a thumbnail picture formed ofthat picture. The selected picture display 103 shows an XGA picture ofone frame, for example. The selected picture display 103 thus presentsan extremely high resolution picture. An unclear image, if displayed onthe whole picture, becomes clear on the selected picture.

FIG. 7 is a diagram illustrating an angle of view of one frame when thecamera unit 3 having a telephoto lens of a magnification of 75 isphotographing. If an object is spaced away from the camera unit 3 by 100m, the one frame covers an area of a vertical dimension of 8.7 m by ahorizontal dimension of 1.17 m. For example, if the image pickup deviceof the camera section 5 uses an XGA format, a single pixel covers anarea of a vertical dimension of 0.87 cm by a horizontal dimension of1.17 cm of the object.

If the object is spaced away from the camera unit 3 by 200 m, the oneframe covers an area of a vertical dimension of 1.74 m by a horizontaldimension of 2.34 m. For example, if the image pickup device of thecamera section 5 uses an XGA format, a single pixel covers an area of avertical dimension of 1.74 cm by a horizontal dimension of 2.34 cm ofthe object.

If the object is spaced away from the camera unit 3 by 500 m, the oneframe covers an area of a vertical dimension of 4.36 m by a horizontaldimension of 5.84 m. For example, if the image pickup device of thecamera section 5 uses an XGA format, a single pixel covers an area of avertical dimension of 4.36 cm by a horizontal dimension of 5.84 cm ofthe object.

A data management method of the captured picture data stored in thearchive 10 or the main memory 30 is discussed below with reference toFIGS. 8A and 8B. As already discussed, at every predetermined intervals,the M×N frames of picture are photographed, compressed, and then stored.As shown in FIG. 8A, the position of each frame is defined by one of theM rows and one of N columns. For example, a position address (1,1)defines the topmost and rightmost frame. Each frame has a filename of aposition address and information about time of recording. The timeinformation is composed of the year, month, day, hour, minute, andsecond. The filename of each frame includes the year, month, day, hour,minute, and second, and the position address.

As shown in FIG. 8B, a direction file is created when the M×N framesform a single whole picture. The direction file defines a set of M×Nframes by including the same data as filename (the year, month, day,hour, minute, and second, and the position address) of a frame havingthe position address (1,1). The direction file contains the positioninformation and metadata of the set of frames. The position informationand metadata are generated by the metadata generator 29. Specifically,the position information (information such as latitude and longitude,bearing, and altitude), and metadata information (time and parameters ofthe camera section 5 such as magnification, focus value, and irisvalue).

The process of capturing and displaying the coverage area picture on thecoverage area picture display 101 will be discussed. With reference to aflow diagram shown in FIG. 9, the picture data is captured into the mainmemory 30 under the control of the controller CPU 33. When the cameraunit 3 is installed at a predetermined place, the pictures are captured.To start this process, a start command is input using a setting menuscreen (not shown). The coverage area picture is captured at any timesuch as at an initial setting.

In step S11, a photographing operation starts at the origin. The originis at the end of the coverage area or at the center of the coveragearea. With the pan part 4 a and tilt part 4 b in the camera unit 3controlled, the optical axis of the lens of the camera unit 3 is alignedwith the photographing direction of the (still) frame at the origin.When one frame is photographed at the origin, the tilt angle andphotographing direction are varied to photograph a next frame. Framesare thus photographed one after another. The photographing direction ofthe camera is varied within the maximum pan angle and the maximum tiltangle.

In step S12, the captured still frame pictures are converted into JPEGdata by the JPEG encoder and metadata attacher 27. In step S13, themetadata and position information are attached to the JPEG data. Themetadata includes the time information, latitude and longitude, etc.,produced by the metadata generator 29, and the position information isthe position address of each frame.

In step S14, the JPEG data, and the metadata and position data attachedthereto are stored onto the main memory 30. With the camera panned andtilted within the maximum range, a number of frames are acquired withinthe maximum range. All frames within the coverage area are thuscaptured, and converted to the JPEG data. The JPEG data, and themetadata and position information attached thereto are stored into themain memory 30. The capturing process of the picture image is thuscompleted. Since the coverage area picture serves as a guide todetermining the photographing direction, a compressed picture or athumbnail may be stored in the main memory 30 rather than storing thesource picture.

The display process of the stored pictures on the coverage area picturedisplay 101 in the display 2 is discussed below with reference to a flowdiagram shown in FIG. 10. The display process is carried by the mainmemory 30, graphic controller 31, image compressor 32 and other blocks.In step S21, picture data retrieved from the main memory 30 isreproduced, and is then subjected to data compression such as datadecimation. The coverage area picture as the thumbnail is thusgenerated. In step S22, the coverage area picture is aligned in positionto be presented on the coverage area picture display 101. In step S23,the thumbnail, namely, the coverage area picture is displayed.

The process of displaying the whole picture on the whole picture display102 is discussed with reference to FIG. 11. The displaying process ismainly carried out by the graphic controller 31. When an arbitrary pointor area within the coverage area picture shown in the above-referencedcoverage area picture display 101 is indicated by a pointer, a controlalgorithm illustrated in the flow diagram shown in FIG. 11 is invoked.

In step S31, the capture position within the coverage area picture isdesignated by the pointer, and the capture coordinates of the wholepicture are verified. For example, the capture position is designated bymoving the line segments 101 a and 101 b shown on the coverage areapicture display 101. Alternatively, the capture position may bedesignated by moving a cursor with a mouse. In step S32, a startposition of the whole picture is calculated. Based on the result ofcalculation, the pan part 4 a and tilt part 4 b in the camera unit 3 arecontrolled. The lens optical axis of the camera unit 3 is shifted to thecapture start position, for example, to a frame at a predeterminedposition from among the set of M×N frames.

In step S33, a still picture photographed by the photographing unit 24is captured as a first frame. In step S34, the still picture data isconverted into JPEG data. In step S35, the metadata and positioninformation are attached to the JPEG data. The conversion of the picturedata into the JPEG data and attachment of the metadata and positioninformation to the JPEG data are performed by the JPEG encoder andmetadata attacher 27.

In step S36, the JPEG data, and the metadata and position informationattached thereto are recorded onto the main memory 30. In step S37, datareproduced from the main memory 30 is displayed at a designated addressin the whole picture display 102 on the display 2 under the control ofthe graphic controller 31.

In step S38, a distance to a photographing position of a next frame iscalculated. In step S39, the pan part 4 a and tilt part 4 b arecontrolled in response to the distance calculated in step S38. Thephotographing position is set to the photographing start position of thenext frame.

In step S40, the number of already captured frames is calculated. It isdetermined in step S41 whether the M×N frames are captured. As alreadydiscussed, if a predetermined number of frames, for example, 2×4 frames,or 4×8 frames is set within the M×N frames, for example, 8×16, it isdetermined whether the predetermined number of frames is captured.

If it is determined in step S41 that the number of already capturedframes has reached the designated number of frames, the algorithmproceeds to step S42. The lens optical axis of the camera unit 3 isshifted to the center of the whole picture display 102. If it isdetermined in step S41 that the number of already captured frames hasnot yet reached the designated number of frames, the algorithm loops tostep S33 to start over with the capturing of a next frame.

The process steps (steps S38 and S39) required to move the photographingposition to capture the next frame may be carried out only when it isdetermined that the number of captured frames has not yet reached thedesignated number.

When any arbitrary point or area within the coverage area picture isdesignated by the pointer, the M×N frames with respect to the designatedposition are captured and the whole picture is then displayed. Thepicture at a point or area designated within the whole picture ispresented on the selected picture display 103 as a selected picture. Theprocess of capturing and displaying the selected picture is carried outby the graphic controller 31 and controller CPU 33 in accordance with aflow diagram shown in FIG. 12.

In step S51, the cursor is moved to a select point on the whole picture,and the mouse is clicked. In step S52, the clicked point is convertedinto position coordinates. The position coordinates are defined for thephotographing area composed of the M×N frames. In step S53, the distancefrom the current photographing position to the designated position iscalculated.

In step S54, the pan part 4 a and tilt part 4 b are controlled to movethe photographing position by the calculated distance. In step S55, aframe is photographed at that position. In step S56, frame data istransferred to the JPEG encoder and metadata attacher 27. The framecaptured by the graphic controller 31 is then presented on the selectedpicture display 103 as a selected picture. The selected picture has thenumber of pixels defined by the XGA format, and is based on uncompresseddata. The selected picture, having a resolution higher than the wholepicture, is clear. Since the selected picture has a size larger than oneframe within the whole picture, the selected picture display 103 thuspresents an expanded picture.

The present invention is not limited to the above-referenced embodiment,and changes and modifications are possible without departing from thescope of the present invention. For example, the maximum pan range is180° in the above-referenced embodiment. The maximum pan range may be360°. The number of coverage area pictures is not limited to one. Aplurality of coverage area pictures are acceptable.

The frame capturing operation of the whole picture in the whole picturedisplay 102 will be discussed now. FIG. 13 is a flow diagramillustrating a frame capturing operation of a frame of the whole picturein accordance with the embodiment of the present invention. If the LIVEMODE button 107 is designated by the pointer, and if the REC button 121is designated by the pointer, a control algorithm represented by theflow diagram is invoked.

When the capture position on the coverage area picture presented on thecoverage area picture display 101 is designated by the pointer in stepS101, the location of the whole picture with respect to the coveragearea picture is determined. The capture coordinates of the whole pictureare thus verified.

In step S102, the capture start position of the whole picture iscalculated. Based on the result of calculation, the pan part 4 a andtilt part 4 b in the camera unit 3 are controlled to move the lensoptical axis of the camera unit 3 to the capture start position. Thecapture start position is the center position of the frame capturedfirst.

In step S103, the lens unit 22, focus-zoom-iris controller 23, andphotographing unit 24 in the camera unit 3 are controlled to capture theframes and to feed the captured frames to the control computer 1 as thepicture data.

In step S104, the picture data supplied from the camera unit 3 isconverted into predetermined picture format data such as JPEG data.

In step S105, the metadata and position information are attached to thepredetermined picture format data.

In step S106, the picture data, and the metadata and positioninformation attached thereto are stored in the main memory 30.

In step S107, the picture data in the predetermined picture format isdisplayed at the designated address, for example, at (0,0) in the wholepicture display 102.

In step S108, the distance of the lens optical axis of the camera unit 3to a next frame is calculated.

In step S109, the pan part 4 a and tilt part 4 b are controlled inaccordance with the distance calculated in step S108, thereby directingthe lens optical axis of the camera unit 3 to the center of the nextframe.

In step S110, the number of captured frames is calculated. For example,a count of a counter may be incremented by one each time one frame iscaptured. The number of frames is thus counted.

In step S111, it is determined whether the counted number of capturedframes has reached the designated number of frames. If it is determinedthat the number of captured frames has reached the designated number offrames, the algorithm proceeds to step S112; otherwise, the algorithmloops to step S103. The designated number of frames is precalculated inaccordance with the mode selected in the REC MODE selection menu 118.Specifically, if the RUN (8×16) mode is selected, the number of framesis 128. If the RUN (4×8) is selected, the number of frames is 32. If theRUN (2×4) is selected, the number of frames is 8.

In step S112, the distance between the current position of the lensoptical axis of the camera unit 3 and the capture start position of thewhole picture display 102 is calculated.

In step S113, the pan part 4 a and tilt part 4 b are controlled based onthe distance calculated in step S112 to direct the lens optical axis ofthe camera unit 3 to the center of the frame serving as the capturestart position.

In step S114, it is determined whether the number of updates of thewhole picture display 102 has reached the predetermined number ofupdates. Specifically, it is determined whether the SELECT mode or RUNmode is selected in the REC MODE selection menu 118. If it is determinedthat the SELECT mode is selected in the REC MODE selection menu 118, thealgorithm proceeds to step S115. If it is determined that the RUN modeis selected in the REC MODE selection menu 118, the algorithm proceedsto step S117.

If the SELECT mod is selected in the REC MODE selection menu 118, thepredetermined number of updates is “1”. All frames presented on thewhole picture display 102 are captured, stored, and then displayed inonly one cycle. Capturing, storage, and displaying of the frames are notrepeated. In contrast, if the RUN mode is selected in the REC MODEselection menu 118, the number of updates is “infinite”. The capturing,storage, and displaying of the frames are repeated until the capturingoperation ends, i.e., until the STOP button 123 is pressed.

In step S115, the distance between the capture start position of thewhole picture display 102 and the center of the whole picture display102 is calculated. Based on the result of calculation, the pan part 4 aand tilt part 4 b are controlled to move the lens optical axis of thecamera unit 3 to the center of the whole picture display 102. The centerof the whole picture means the center position of the 8×16 frames, forexample.

In step S116, the operation of the stepping motors of the pan part 4 aand tilt part 4 b is suspended. The control algorithm represented by theflow diagram thus ends.

In step S117, it is determined whether the end command of the capturingoperation is issued. Specifically, it is determined whether the STOPbutton 123 is designated by the pointer. If it is determined that theSTOP button 123 is designated by the pointer, the algorithm proceeds tostep S115. If it is determined that the STOP button 123 is notdesignated by the pointer, the algorithm loops to step S103.

FIG. 14 is a flow diagram illustrating a reproduction operation ofstored picture data in accordance with one embodiment of the presentinvention. If the VIEW MODE button 106 is designated by the pointer, andif the PLAY button 122 is designated by the pointer, the controlalgorithm represented by the flow diagram is invoked.

If the PLAY button 122 is designated by the pointer in step S201, therecorded data display screen appears in a pop-up window shown in FIG. 5,for example.

In step S202, it is determined whether the date is designated in thedate box 164 and whether the time is designated in the time box 165. Ifit is determined that the data and time are respectively designated inthe date box 164 and time box 165, the algorithm proceeds to step S203.If it is determined in step S202 that no date is designated in the datebox 164 with no time designated in the time box 165, or if it isdetermined in step S202 that either the date or the time is notdesignated in the date box 164 or the time box 165 respectively, stepS202 is repeated until both the data and the time are designated in thedate box 164 and the time box 165, respectively.

In step S203, the coverage area picture and/or the whole picture arepresented on the coverage area picture display 101 and/or the wholepicture display 102 based on the recorded data at the designated dateand time. The algorithm of the flow diagram then ends.

When the reproduction operation is retrospectively performed from thecurrent time to past time, the VIEW MODE button 106 is designated by thepointer. The RUN mode is selected in the REC MODE selection menu 118,and the PLAY button 122 is designated by the pointer. In subsequentsteps, the date and time of the recorded data with which thereproduction operation starts, and the date and time of the recordeddata with which the reproduction operation ends are designated. In thisway, data captured at the designated starting date and time to datacaptured at the designated ending date and time can be reproduced. It isalso possible to reproduce recorded data in the order from a past pointof time to current time.

FIG. 15 is a flow diagram illustrating a capturing operation of oneframe only at a designated arbitrary position. If the LIVE MODE button107 is designated with the pointer and if an arbitrary position in thewhole picture display 102 is designated with the pointer, a controlalgorithm of the flow diagram is invoked.

If the SELECT button 117 is designated by the pointer in step S301, theSELECT display shown in FIG. 4 appears in a pop-up window format, forexample.

In step S302, the whole picture presented on the whole picture display102 is also presented on the SELECT display of the display screen 152.The picture presented on the display screen 152 has frame border linesalong which each frame is captured. The whole picture presented on thewhole picture display 102 may be shown segmented by the unit of framesaccording to which the whole picture is captured, on the display screen152. Also, a grid of lines may be shown superimposed on the wholepicture.

In step S303, a desired frame on the display screen 152 is designatedusing the pointer.

In step S304, it is determined where the selected frame is located inposition within the display screen 152. The position of the selectedframe is thus verified.

In step S305, for example, luminance of the selected frame is varied toallow the selected frame to be easily recognized on the display screen152. Any means is acceptable as long as the selected frame isrecognized. For example, the selected frame may be shown with a colordifference signal thereof varied, with any or all of RGB signals thereofvaried, with the color of an outline thereof changed, or with theoutline thereof blinked.

In step S306, it is determined where the selected frame with the displaythereof varied is located in position within the coverage area picturedisplay 101, and the coordinates of the selected frame are thusverified.

In step S307, the closing buttons 151 and 153 are designated using thepointer, and the SELECT display is closed.

In step S308, the pan part 4 a and tilt part 4 b are controlled to movethe lens optical axis of the camera unit 3 to the center position of theselected frame.

In step S309, the lens unit 22, focus-zoom-iris controller 23, andphotographing unit 24 in the camera unit 3 are controlled to capture theframes and to feed the captured frames to the control computer 1 as thepicture data.

In step S310, the picture data supplied from the camera unit 3 isconverted into predetermined picture format data such as JPEG data.

In step S311, the metadata and position information are attached to thepredetermined picture format data.

In step S312, the picture data, and the metadata and positioninformation attached thereto are stored in the main memory 30.

In step S313, the picture data in the predetermined picture format isdisplayed at the designated address in the whole picture display 102.

In step S314, it is determined whether the number of updates of thesingle selected frame has reached the predetermined number of updates.Specifically, it is determined whether the SELECT mode or RUN mode isselected in the REC MODE selection menu 118. If it is determined thatthe SELECT mode is selected in the REC MODE selection menu 118, thealgorithm proceeds to step S315. If it is determined that the RUN modeis selected in the REC MODE selection menu 118; the algorithm proceedsto step S317.

If the SELECT mod is selected in the REC MODE selection menu 118, thepredetermined number of updates is “1”. The single selected frame iscaptured, stored, and then displayed in only one cycle. Capturing,storage, and displaying of the frame are not repeated. In contrast, ifthe RUN mode is selected in the REC MODE selection menu 118, the numberof updates is “infinite”. The capturing, storage, and displaying of theframes are repeated until the capturing operation ends, i.e., until theSTOP button 123 is pressed.

In step S315, the distance between the capture position of the singleselected frame and the center of the whole picture display 102 iscalculated. Based on the result of calculation, the pan part 4 a andtilt part 4 b are controlled to move the lens optical axis of the cameraunit 3 to the center of the whole picture display 102. The center of thewhole picture means the center position of the 8×16 frames, for example.

In step S316, the operation of the stepping motors of the pan part 4 aand tilt part 4 b is suspended. The control algorithm represented by theflow diagram thus ends.

In step S317, it is determined whether the end command of the capturingoperation is issued. Specifically, it is determined whether the STOPbutton 123 is designated by the pointer. If it is determined that theSTOP button 123 is designated by the pointer, the algorithm proceeds tostep S315. If it is determined that the STOP button 123 is notdesignated by the pointer, the algorithm loops to step S309. FIG. 16 isa flow diagram illustrating an operation in which one frame only at anarbitrary position is reproduced from the recorded data in accordancewith the embodiment of the present invention. An algorithm of the flowdiagram shown in FIG. 16 is invoked if the VIEW MODE button 106 isdesignated using the pointer, if the RUN mode is selected in the RECMODE selection menu 118, and if any arbitrary position within the wholepicture display 102 is designated using the pointer.

If the SELECT button 117 is designated by the pointer in step S401, theSELECT display shown in FIG. 4 appears in a pop-up window format, forexample.

In step S402, the whole picture presented on the whole picture display102 is also presented on the SELECT display of the display screen 152.The picture presented on the display screen 152 has frame border linesalong which each frame is captured. The whole picture presented on thewhole picture display 102 may be shown segmented by the unit of framesaccording to which the whole picture is captured, on the display screen152. Also, a grid of lines may be shown superimposed on the wholepicture.

In step S403, a desired frame on the display screen 152 is designatedusing the pointer.

In step S404, it is determined where the selected frame is located inposition within the display screen 152. The position of the selectedframe is thus verified.

In step S405, for example, luminance of the selected frame is varied toallow the selected frame to be easily recognized on the display screen152. Any means is acceptable as long as the selected frame isrecognized. For example, the selected frame may be shown with a colordifference signal thereof varied, with any or all of RGB signals thereofvaried, with the color of an outline thereof changed, or with theoutline thereof blinked.

In step S406, it is determined where the selected frame with the displaythereof varied is located in position within the coverage area picturedisplay 101, and the coordinates of the selected frame are thusverified.

In step S407, the recorded data display screen illustrated in FIG. 5appears in a pop-up window format.

In step S408, the date and time of recorded data with which areproduction operation starts are designated. For example, the date ofthe recorded data with which the reproduction operation starts may bedesignated in the date box 164, and the time of the recorded data atwhich the reproduction operation starts may be designated in the timebox 165. Desired recorded data may be selected from among the recordeddata presented on the recorded data display area 166.

In step S409, the data and time of recorded data with which thereproduction operation ends are designated. For example, the date of therecorded data with which the reproduction operation ends may bedesignated in the date box 164, and the time of the recorded data atwhich the reproduction operation ends may be designated in the time box165. Desired recorded data may be selected from among the recorded datapresented on the recorded data display area 166. If the date and time ofthe recorded data with which the reproduction operation ends are notdesignated, all recorded data which is stored from the starting date andtime are reproduced in retrospect.

In step S410, the closing button 151 or 153 is designated using thepointer, and the SELECT display is closed.

In step S411, the recorded data to be reproduced is read from the mainmemory 30. Alternatively, the recorded data may be read from the archive10. Since a plurality of frames are stored in the archive 10 as onearchive, the recorded data is decompressed, and then read.

In step S412, the frame at the selected coordinates in the read data isdisplayed at the selected coordinates on the whole picture display 102.

In step S413, it is determined whether it is the end date and time toend the reproduction operation. If it is determined that it is the enddate and time to end the reproduction operation, the algorithm ends. Ifit is determined that it is not the end date and time, the algorithmproceeds to step S414.

In step S414, recorded data to be reproduced next is read.

The present invention is not limited to the above embodiment. Variouschanges and modifications are possible without departing from the scopeof the present invention.

It is possible to reproduce, in retrospect, the data recorded fromcurrent time to a past point of time. It is also possible to reproducethe data recorded from a past point of time to current time.

The control computer 1 connected to the LAN 7 controls the camera unit 3in the system. Only both the control computer 1 and the camera unit 3may be of a mobile type.

In the above-referenced embodiment, the frames at any positions areconsecutively photographed, stored, and displayed. Alternatively, theframes at any positions may be photographed, stored, and displayed atpredetermined intervals.

In the above-referenced embodiment, the frames at any positions areconsecutively photographed, stored, and displayed. Alternatively, theframes at any positions may be only photographed and displayed, but notbeing stored.

In the above-referenced embodiment, the frames at any positions areobtained from the recorded data in accordance with position coordinates.Alternatively, the frames at any positions may be obtained from therecorded data referencing the position information and/or the metadataattached to the frames.

In the above-referenced embodiment, the camera unit 3 is tilted downwardto successively photograph the frames. Alternatively, the camera unit 3may be tilted upward to successively photograph the frames. The cameraunit 3 may be panned clockwise or counterclockwise to successivelyphotograph the frames.

In the above-referenced embodiment, the frame at any arbitrary positionis obtained from the recorded based on the position coordinates.Alternatively, the frames may be respectively numbered with referencenumbers 1, 2, 3, . . . from the home position for identification, and aframe at any position may be obtained from the recorded data accordingthe reference number.

The period of time required to capture the whole panorama picture isprevented from being prolonged because the picture photographing unit isnot fully moved within the predetermined range. Since the coverage areapicture with the picture photographing unit fully moved within thepredetermined range is displayed, the photographing direction to obtainthe picture of a desired area is easily set. Even if the picture beingcaptured is dark, the photographing direction is easily set. Theoperability of the system is improved.

In accordance with the present invention, the entire picture within anarea to be monitored is displayed, and the frame at any position isphotographed, and displayed. The frame at any position is displayed indetail.

In accordance with the present invention, the entire picture within anarea to be monitored is displayed, and the frames at any positionacquired in the past are displayed in retrospect in the order fromcurrent time to a past point of time. The frame at any position isdisplayed in detail.

In accordance with the present invention, the entire picture within anarea to be monitored is displayed, and the frames at any positionacquired in the past are displayed in retrospect in the order from apast point of time to current time. The frame at any position isdisplayed in detail.

1-32. (canceled)
 33. A signal processing apparatus comprising: a storeconfigured to store a whole image from which plural images to betransmitted are extracted; an input configured to receive a currentlyphotographed whole image from which plural images to be transmitted areextracted; a first encoding section configured to encode a first imagecomprised of the stored or received whole image by a first encodingprocess with a first resolution, the encoded first image being decodableby a first decoding process; a second encoding section configured toencode a second image comprised of at least one of the plural frames ofthe first image by a second encoding process with a second resolutionhigher than the first resolution, the encoded second image beingdecodable by a second decoding process; a setting section configured toset an area in the first image in which the at least one of the pluralframes is located and select, for encoding as the encoded second image,the at least one of the plural frames in the set area; and an outputsection configured to output the first and second encoded images fordecoding; wherein the first encoding process is different from thesecond encoding process.
 34. A signal processing apparatus according tothe claim 33, wherein the first encoding process includes imagecompression to compress image data more than the second encodingprocess.
 35. A signal processing apparatus according to the claim 33,wherein the output section is controlled such that when the first andsecond images are displayed, the second image is displayed larger thanthe first image.
 36. A signal processing apparatus according to theclaim 33, wherein the setting section is configured to change at leastone of brightness, resolution and contrast of the second image.
 37. Asignal processing apparatus according to the claim 33, wherein thesetting section is configured to set a changeable color border for thearea of the second image.
 38. A signal processing apparatus according tothe claim 33, wherein the first image is generated by synthesizingplural images.
 39. A signal processing apparatus according to the claim33, wherein the signal processing apparatus is used for a surveillancesystem including plural cameras.
 40. A signal processing apparatuscomprising: circuitry configured to: store a whole image from whichplural images to be transmitted are extracted; receive a currentlyphotographed whole image from which plural images to be transmitted areextracted; encode a first image comprised of the stored or receivedwhole image by a first encoding process with a first resolution, theencoded first image being decodable by a first decoding process; encodea second image comprised of at least one of the plural frames of thefirst image by a second encoding process with a second resolution higherthan the first resolution, the encoded second image being decodable by asecond decoding process; set an area in the first image in which the atleast one of the plural frames is located and select, for encoding asthe encoded second image, the at least one of the plural frames in theset area; and output the first and second encoded images for decoding;wherein the first encoding process is different from the second encodingprocess.
 41. A signal processing apparatus according to the claim 40,wherein the first encoding process includes image compression tocompress image data more than the second encoding process.
 42. A signalprocessing apparatus according to the claim 40, when the first andsecond images are displayed, the second image is displayed larger thanthe first image.
 43. A signal processing apparatus according to theclaim 40, wherein the circuitry is configured to change at least one ofbrightness, resolution and contrast of the second image.
 44. A signalprocessing apparatus according to the claim 40, wherein the circuitry isconfigured to set a changeable color border for the area of the secondimage.
 45. A signal processing apparatus according to the claim 40,wherein the first image is generated by synthesizing plural images. 46.A signal processing apparatus according to the claim 40, wherein thesignal processing apparatus is used for a surveillance system includingplural cameras.
 47. A signal processing method comprising: storing awhole image from which plural images to be transmitted are extracted;receiving a currently photographed whole image from which plural imagesto be transmitted are extracted; encoding a first image comprised of thestored or received whole image by a first encoding process with a firstresolution, the encoded first image being decodable by a first decodingprocess; encoding a second image comprised of at least one of the pluralframes of the first image by a second encoding process with a secondresolution higher than the first resolution, the encoded second imagebeing decodable by a second decoding process; setting an area in thefirst image in which the at least one of the plural frames is locatedand selecting, for encoding as the encoded second image, the at leastone of the plural frames in the set area; and outputting the first andsecond encoded images for decoding; wherein the first encoding processis different from the second encoding process.